We formulate grasp learning as a neural field and present Neural Grasp Distance Fields (NGDF). Here, the input is a 6D pose of a robot end effector and output is a distance to a continuous manifold of valid grasps for an object. In contrast to current approaches that predict a set of discrete candidate grasps, the distance-based NGDF representation is easily interpreted as a cost, and minimizing this cost produces a successful grasp pose. This grasp distance cost can be incorporated directly into a trajectory optimizer for joint optimization with other costs such as trajectory smoothness and collision avoidance. During optimization, as the various costs are balanced and minimized, the grasp target is allowed to smoothly vary, as the learned grasp field is continuous. In simulation benchmarks with a Franka arm, we find that joint grasping and planning with NGDF outperforms baselines by 63% execution success while generalizing to unseen query poses and unseen object shapes. Project page: https://sites.google.com/view/neural-grasp-distance-fields.

我们提出了Theseus，这是一个有效的应用程序不合时宜的开源库，用于在Pytorch上构建的可区分非线性最小二乘（DNL）优化，为机器人技术和视觉中的端到端结构化学习提供了一个共同的框架。现有的DNLS实施是特定应用程序的，并且并不总是纳入许多对效率重要的成分。 Theseus是应用程序不可静止的，正如我们使用的几个示例应用程序所用的，这些应用程序是使用相同的基础可区分组件构建的，例如二阶优化器，标准成本功能和Lie组。为了提高效率，TheseUS纳入了对稀疏求解器，自动矢量化，批处理，GPU加速度和梯度计算的支持，并具有隐式分化和直接损耗最小化。我们在一组应用程序中进行了广泛的性能评估，显示出这些功能时显示出明显的效率提高和更好的可扩展性。项目页面：https：//sites.google.com/view/theseus-ai

3D场景图（3DSG）是新兴的描述；统一符号，拓扑和度量场景表示。但是，典型的3DSG即使在小环境中包含数百个对象和符号。完整图上的任务计划是不切实际的。我们构建任务法，这是第一个大规模的机器人任务计划基准3DSGS。尽管大多数基准在该领域的基准努力都集中在基于愿景的计划上，但我们系统地研究了符号计划，以使计划绩效与视觉表示学习相结合。我们观察到，在现有方法中，基于经典和学习的计划者都不能在完整的3DSG上实时计划。实现实时计划需要（a）稀疏3DSG进行可拖动计划的进展，以及（b）设计更好利用3DSG层次结构的计划者。针对前一个目标，我们提出了磨砂膏，这是一种由任务条件的3DSG稀疏方法。使经典计划者能够匹配，在某些情况下可以超过最新的学习计划者。我们提出寻求后一个目标，这是一种使学习计划者能够利用3DSG结构的程序，从而减少了当前最佳方法所需的重型查询数量的数量级。我们将开放所有代码和基线，以刺激机器人任务计划，学习和3DSGS的交叉点进行进一步的研究。

我们解决了在手动操纵期间从触摸跟踪3D对象姿势的问题。具体地，我们使用基于视觉的触觉传感器来看看追踪小物体，该触觉传感器在接触点提供高维触觉图像测量。虽然事先工作依赖于有关已本地化对象的先验信息，但我们删除此要求。我们的关键识别是，一个对象由几个本地曲面修补程序组成，每个界面都足以实现可靠的对象跟踪。此外，我们可以通过提取嵌入在每个触觉图像中的局部表面正常信息在线恢复此本地补丁的几何形状。我们提出了一种新的两阶段方法。首先，我们使用图像翻译网络学习从触觉图像到曲面法线的映射。其次，我们在因子图中使用这些表面法线到两个重建本地补丁映射并使用它来推断3D对象姿势。我们展示了在唯一形状的100多个联系序列中跟踪可靠的对象跟踪，其中仿真中的四个对象和现实世界中的两个对象。补充视频：https://youtu.be/jwntc9_nh8m

使用视频指定任务是获取新颖和一般机器人技能的强大技术。然而，推理机械和灵巧的互动可以使其挑战规模学习接触的操纵。在这项工作中，我们专注于视觉非预先展示平面操作的问题：给定平面运动中对象的视频，找到再现相同对象运动的联系人感知机器人动作。我们提出了一种新颖的架构，可微分的操纵（\我们）的学习，它通过利用可微分优化和基于有限差分的模拟来将视频解码与接触机械的前沿的视频解码神经模型结合在一起。通过广泛的模拟实验，研究了基于模型的技术与现代深度学习方法之间的相互作用。我们发现，我们的模块化和完全可差的架构比看不见的对象和运动的学习方法更好。 \ url {https://github.com/baceituno/dlm}。

我们解决了学习观察模型的问题，用于估计的结束到底。在部分可观察环境中运行的机器人必须使用捕捉潜在状态和观察之间的联合分布的观测模型来推断潜在的状态。该推理问题可以作为使用所有先前测量的最可能的状态序列优化的图表中的目标。前工作使用观察模型，即已知先验，或者独立于图形优化器的代理损耗培训。在本文中，我们提出了一种方法，通过在循环中使用图形优化器学习观察模型来直接优化端到端跟踪性能。然而，可能出现这种直接方法，要求推断算法完全可分辨率，这很多最先进的图表优化器不是。我们的主要洞察力是推出作为基于能源学习的问题。我们提出了一种新颖的方法，Leo，用于学习观察模型的结束，具有可能是不可差异的图优化器。 Leo在从图形后面的采样轨迹之间交替，并更新模型以将这些样本与地面真相轨迹匹配。我们建议使用增量高斯牛顿溶剂有效地生成这些样品。我们将Leo与来自两个独特任务的数据集上的基线进行比较：导航和现实世界的平面推动。我们表明Leo能够学习具有较低误差和更少样本的复杂观测模型。补充视频：https://youtu.be/yqzlupudfka

我们介绍了栖息地2.0（H2.0），这是一个模拟平台，用于培训交互式3D环境和复杂物理的场景中的虚拟机器人。我们为体现的AI堆栈 - 数据，仿真和基准任务做出了全面的贡献。具体来说，我们提出：（i）复制：一个由艺术家的，带注释的，可重新配置的3D公寓（匹配真实空间）与铰接对象（例如可以打开/关闭的橱柜和抽屉）； （ii）H2.0：一个高性能物理学的3D模拟器，其速度超过8-GPU节点上的每秒25,000个模拟步骤（实时850x实时），代表先前工作的100倍加速；和（iii）家庭助理基准（HAB）：一套辅助机器人（整理房屋，准备杂货，设置餐桌）的一套常见任务，以测试一系列移动操作功能。这些大规模的工程贡献使我们能够系统地比较长期结构化任务中的大规模加固学习（RL）和经典的感官平面操作（SPA）管道，并重点是对新对象，容器和布局的概括。 。我们发现（1）与层次结构相比，（1）平面RL政策在HAB上挣扎； （2）具有独立技能的层次结构遭受“交接问题”的困扰，（3）水疗管道比RL政策更脆。

Wind power forecasting helps with the planning for the power systems by contributing to having a higher level of certainty in decision-making. Due to the randomness inherent to meteorological events (e.g., wind speeds), making highly accurate long-term predictions for wind power can be extremely difficult. One approach to remedy this challenge is to utilize weather information from multiple points across a geographical grid to obtain a holistic view of the wind patterns, along with temporal information from the previous power outputs of the wind farms. Our proposed CNN-RNN architecture combines convolutional neural networks (CNNs) and recurrent neural networks (RNNs) to extract spatial and temporal information from multi-dimensional input data to make day-ahead predictions. In this regard, our method incorporates an ultra-wide learning view, combining data from multiple numerical weather prediction models, wind farms, and geographical locations. Additionally, we experiment with global forecasting approaches to understand the impact of training the same model over the datasets obtained from multiple different wind farms, and we employ a method where spatial information extracted from convolutional layers is passed to a tree ensemble (e.g., Light Gradient Boosting Machine (LGBM)) instead of fully connected layers. The results show that our proposed CNN-RNN architecture outperforms other models such as LGBM, Extra Tree regressor and linear regression when trained globally, but fails to replicate such performance when trained individually on each farm. We also observe that passing the spatial information from CNN to LGBM improves its performance, providing further evidence of CNN's spatial feature extraction capabilities.

Recent advances in deep learning have enabled us to address the curse of dimensionality (COD) by solving problems in higher dimensions. A subset of such approaches of addressing the COD has led us to solving high-dimensional PDEs. This has resulted in opening doors to solving a variety of real-world problems ranging from mathematical finance to stochastic control for industrial applications. Although feasible, these deep learning methods are still constrained by training time and memory. Tackling these shortcomings, Tensor Neural Networks (TNN) demonstrate that they can provide significant parameter savings while attaining the same accuracy as compared to the classical Dense Neural Network (DNN). In addition, we also show how TNN can be trained faster than DNN for the same accuracy. Besides TNN, we also introduce Tensor Network Initializer (TNN Init), a weight initialization scheme that leads to faster convergence with smaller variance for an equivalent parameter count as compared to a DNN. We benchmark TNN and TNN Init by applying them to solve the parabolic PDE associated with the Heston model, which is widely used in financial pricing theory.

In this manuscript, we present a novel method for estimating the stochastic stability characteristics of metastable legged systems using the unscented transformation. Prior methods for stability analysis in such systems often required high-dimensional state space discretization and a broad set of initial conditions, resulting in significant computational complexity. Our approach aims to alleviate this issue by reducing the dimensionality of the system and utilizing the unscented transformation to estimate the output distribution. This technique allows us to account for multiple sources of uncertainty and high-dimensional system dynamics, while leveraging prior knowledge of noise statistics to inform the selection of initial conditions for experiments. As a result, our method enables the efficient assessment of controller performance and analysis of parametric dependencies with fewer experiments. To demonstrate the efficacy of our proposed method, we apply it to the analysis of a one-dimensional hopper and an underactuated bipedal walking simulation with a hybrid zero dynamics controller.